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tty INTRODUCTIGN 


i OVERVIEW 

Many orfices in @the public™=anda private  seetorseso: =) cn 
United States todau are concerned with increasing the 
africienmcy and productivity of the work force. Modern 
ecmnology is often involved in accomplishing these ends. 


Froviding some advanced technology to the worker may, 


NOWEVEAT, cTeate a situation which causes the job to be more 
complex than 16 Was vonriginai lu. and many people are not 
prepared for the new environment. This iS a very common 


problem in the use of computers. There has been 2 push to 


simolifu this mean-machine interface, eng many input and 
OQuULGUT methods have been expiored. With the increasing 
EMoOHasis on man-machine interface and communication, 


avisometic speech recognition (ASR) 1s expected to btecaome a 
Magar ferce. ASR involves communicating with 3 machine. When 
the ASR equipment correctly recognizes what is being spoken, 


it sends out &@ predetermined ASCII string of characters to a 


computer, or it could be used to send a signal to turn off 


lights or urn on power st rehes: The advantages of using 
speech in communicating with a machine include Les 
convenience, speed, Universality, and the possibility of 


Zvaiding or tTeducing the training of workers through its 
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USE SULOnew@mem SPeecherecognituon is considered to be the 


Peete CU oc eemd complex problem in the field of voice 
By oc SSica ng Pao’ Sear ge=scale integration (LSlex and very 
lerge-scale integration (VLSI) chip design, signal 
sracessing SBcousti1e phonetics: matural language theory, 
emg istics: mathematics af steachastic pTOcesses, arid 


comoutrver science techniques CRef. 13. 

Speeai. Seer es metura! form, iss the fastest and .most 
cenmvenient mesns For humans to communicate with each other. 
wWnile humans may be hindered by a language barrier, machines 
cen 62 treined to learn and respond to any language. 
; ASR is convenient in that hands and eyes are 
Free and the speaker has mobility, which other 


Sr Gumeeations modes do not offer. With a mictophone 
headset, the speaker is free to move about and perform 


oy 


Ww) 


Pe@eerren. TrUiTctions such a Vvieuling graphics screen 


rT decision aids “1 order tie make gdecisions: OT 


ye 
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Progress toward economically feasible voice recognition 


systems has been slow: but ASR products are gradualiy 


pecoming avaliable, primarily for industrial applications. 
PweostT let applications include “hands-busy" activities 
invelving inspections usually associated with quality 


cenerel: flight training simulators for entering commands: 
bagagage/percel sorting. voice contro! of machine tooling: 


and home appliance control. Te meonl veg. States Nationa: 


“Tey “Agee CN5A} has developed speech recognition 
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gorithms ailocuwing it Mtge spam teu Words Wtreeave: een vec 
yerbeal transmissions from unfriendly netions. CRef. uae) 


Malitatry applications include aircraft cockpit voice-entered 
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Lomnand ana computet~assisted training of aif Cid le 


gm tro. bei oe 


b PURFOSE OF THE iia ais 


im the Push 60 Inmesrporsalte) emetaindg ~“LeEehna. og ime 
Yepilous fields, there have been yaps in the research to show 
whether or not ASR enhances productivity and efficiency. A 


comprehensive comparison of computer input technologies in a 


Hilitary command and control environment has not been 
pertarmed. This thesis describes an experiment conducted at 
the Naval Postgraduate School (NPS) during the period 39 
mcotocter 19684 to 30 November i984. This experiment was the 
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enpt to compare twe difterentervolce Recoani won 


><? 


aft 


= 
ft 
cf 


ems of differing technologies and keuboard (typing) 


mods 
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of input in issuing commands in a controlled wargame. 
The principle objective of the thesis was to design ana 
molement en experiment in which the three modes of input 
could be utilized, resulting in data that would be used ta 
determine S&S measure of performance to make a direct 
camoarison of the devices. The authors became involved iin 


the project during the conceptual stage, and performed the 


design, setup, conduct, and analysis of the experiment. The 


GSOai of the experiment was to develop wargame scenarios that 


eric sSimvlate military conflict Situations Pequar 2nd 
gecs.sion making and lssuance or orders under time 
coamstreints. There had to be sufficient replications of 


te tels: controi sof procedures, and Limitation of variables 
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(eave toticaiet significant conclusions. 
Mumer ous experiments concerning automatic speech 
Bee iricien fave been conducted at NPS. This experiment was 
eo sect COMmeang: GOontrel, and Communications (C3) 
So er ator u: also known as the Wargaming Anéiusis and 
neseerch (WER) Laboratory, using a VAX i1/78B0 computer buiit 
iy Digital Equipment Corporation. The Naval Warfare 


mg lation: sSustem  <“NWiSs), a computer based 
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used to simulate military command and 


moral situations for students serving as subjects for the 


experiment. fhe two types of speech recognition equipment 
availsble in the CS Lab were the Verbex 3OOG continuous 


rput system and the Threshold T6G0 discrete speech 
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syaetem. A Digital VTL1O2 terminal, including a2 video screen 
and keyboard, was used for the typing mode of input to the 
compeuter. A gtoup of students was assigned to each of the 
three input devices. Different scenarios and environments 
were designed for the experiment, but all three command 
imput methods were used for each combination of scenario and 


environment in order to determine which input mode was best 


in different situations. 


2 SUMMARY 

This thesis pPravices @ GeEScTIntign Qo thew ex gerinen. 
thar was conducted, including background materiel ana 
procedures leading up to the experiment. Basic speech 


recognition technology is described. and a review of past 
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riments with speech recognition at NPS 15 provided. The 


cgesign process for this experiment is outiined, including 


the gesign of the scenarios, determination of environments 
peer be used, and the tackground of the subyects who 
paTticipeted in this project. A Geseription of the Vowec 
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‘@ining processes, practice sessions, and actual conduct of 


‘he experiment 1s provided. Analysis of the data from the 
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xperiment 1S provided=imn Chapter Vives inaliu, cone luUstens 
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the experiment and the authors’ recommendations for 


Further studu and applications follow the analysis of the 
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AUTOMETIC speech recognition technology has progressed 


Moet e PSInt Where Voice Tecogn@tion products are being used 


im some anoliceations, But further edvances must be made 
aefore machines that recognize speech will become 
commonolace. This chapter will discuss some terminology used 


Poe ot te “Tecognition, explain some general theory in ASR, 
review wotk that hes been done at tne Naval Postgraduate 
Schaci in this field, ang describe the equipment that was 


ywsed in this experiment. 
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oe MereanlLiiuN OF TERMS 
& distinction tas to be made between recognition anda 


Pmgerstancing itt automatic soeech Tecogul tion. Pose 


if 


eccessful voice recognition machines today recognize short 
mecete@ ces ar Speech. 4m utterance refers to words or short 
ohrtases spoken with pauses between them. Recognition cf 


hort utterances using pattern matching techniques is 
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pattern recognition, not understanding. W. A. Lea defines 
voice recognition bu machine “generally as the process of 
transforming the comtinuous fuman acoustic signal into 
gi¢crete representations which may be comprehended to affect 


Teamoemsive behsvicor." CRef. 2: p. 40] 


ne technolsgy required for reeoagnition “oe, eegenaae 


cCumversationa} speech exceeds current capabilities. The 
aqifFficuity Liles IR Che Contests. Hence hee Ss CO eee ene In 
canversational speech, individual words often do not carry 
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mh antormation for the word to be recognized by itself. 
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mteins many acoustically ambiguous sounds, ¢0 that 
information needea to interpret 4 spoken word is not found 
.mf the word itself approximately thirty percent of the time. 


ic iNToTmMation if Spread ot over sevetal words rather than 


conteined in aindivicual phonemes. & phoneme 1s a 
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member af the set of smallest units of speech. Thus 


On be meus & information processing may be required for any 


‘ch recognition beyond that of a tlimited number of 
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carefully pronounced isolated utterances. Artif i Cage 
inte®iligence and contextual information processing are key 
Boe the FU toRe development ang advances oft speech 
PSQegni Vien, Cee 


There ere two Mejor CateGOrTices Se. se epeten Teengritprag 


“used. Discrete speech (isolated utterances) refers to words 
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nert phrases that are spoken with pauses between them. A 


Sap of at igezet O.1 seconds of silence between utterances is 
required in order to distinguish between the boundaries of 
the utterances. Continuous speech refers to normal speech 
containing no word boundaries or pauses. Systems that take 


ore, 


comtinuous speech and attempt to use semantic information 


are called understanding systems. Within both discrete and 
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continuous Categories, there are speaker-independent systems 
and speaker-dependent systems. With speaker-dependent 
systems, individual users must "train" the system with a 


fewemeset Of YetCabulary in order for the system to recognize 
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CURRENTLY FIELDED 


_-_-- UNDER DEVELOPMENT 


Raoure Zain oR Systemic 


theit voices. Training involves users giving representative 
samples of their voice for each utterance. With speaker- 
independent systems, individual users are not required to 


train their voice patterns. They use the vocabulary already 


ie 


-torec inf memory which may be used by any user. At this 


Time, there are no continuous speaketr-independent systens on 


ae -ROBLEMS OP (SPSSGr yee eos ei ai. 


The mein gual of researchers in the automatic speech 


recogmition Fieid 15 to develop systems that wili recognize 
speek with the greatest degqree oF accuracy, in the least 


amounts of time, and at the lLawest cost. The more advafriced an 


Sucometic speech recognition system is, and the larger its 
vocabulary: the greater the challenge to develop more 
afticient methods to recognize voice inputs. Gne problem igs 


how t9 combine storage and computation considerations in 


OPGer to Gptimize syustem performance. There are aifferent 
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or computational advantages in using sets of TuUleEs, 
dures, templates, or networks to improve oerfermance. 


Sigarithm improvements aimed at efficient search stratesies 


Ft 
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eng orecomputing relationship networks are two methods of 
Speeding up the recognitiscn process, rather than using 
randcam or exheustive search techniques. These methods can 
aiso improve Tecogn? tre mean euro c Y- 


Uther probiems ate concerned with machine intelligence. 
These include how to represent and utilize contextual 
information, and how to make a classification decision using 


advice from acoustic and semiotic (non-acoustic?) knowledge 
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Automatic speech recognition equipment is composed 


euumad atialUuzers Followed by word classifiers There are 


Zev oe ope cai TetCognition which consist of é 
(saucer: signal processor: PeaulUTe 6% crac Tor, and 
eomee clessirier 

& Sound Analysis 


he transducer end etignsei prscessor eguate to 


and cochles. these are moet commoniu a 
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ophone end spectrum analyzer. They take the voice analog 


mei and measure the relative intensities of the component 


waves. Acoustic information 15 Tepresentec by the time 


ution of the power spectrum. The initial analyses 
uCce perametric representations of the speech. The 
meer gr the feature extractor is basically that of data 
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om. The parametric Tepresentation of the speech may 
ein redundant or irrelevant information which can be 
ved without losing any information. The removal of this 
evans rot matiom reduces the computational load for 
subsequent processing. CRef. SJ 
b. Crass) 1 Gact on 

The utterance classifier equates to the brain. 


this steo, internal models of acoustic patterns produced 


seech are stored in memory. These reference patterns are 
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Matched SgGainst the internal representations o the spoken 
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Piscrete Speech Recognition 
The differences between discrete and continuous 
speach systems become obscure when attempting to classify a 


SaAts 
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ct 


iculsr approach as either discrete or continuous. The 


most oovicus characteristics of discrete Speech are the 
TERULLTESad Feuses between utterences and the speaking rates of 
Uae TS. Average speaking Tates between thirty and seventy 


wtberances per minute have been achieved by individvate. 


usiNG discrete speech in factory environments. CRef. 4: p. 
174] 

The most elementary and widely used approach to 
speech clessification is the use of a two-dimensional matrix 
of the utterance; with time along one axis and frequency 


con Geme aleiug tive other axis. Refsrence matrices 
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ah.cn are genetated for each word in the vocabulary during 


isar training are compared ta the matrix of the inceming 
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his comparison is tupically performed by computing 
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sum of the square of the Gistances between corresponding 


Metrix elements in the reference and unknown matrices. ECRef. 


There is commonly a distortion between an utterance 
and its tTreference. This is partially due to 2 non-linear 
waTping of the time scele, caused by the word being 


stonounced at & different speed than its reference. To find 
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the oOptimel warping of the time scale, the speech 18 divided 


tnto 6 6©6Staendcard frames of time Ccommoniy ten msec). Cine 


method of time a2iignment is to use linear time scaling. Deta 


cCoTresponaing to each frame is stretched ofr compressec 
Linesrig sao that the utterance and Teference are the same 
Benesch, With the beginnings and ends being matched. CRef. 5S: 
eee. Pe 


One of the limitations with template matching is 


fi 


thet botn storage and computation grow linearly with the 


Size cf the vocabulary. Discrete speech sustems are 
tupirceilu limited to SO to 200 utterances, though some may 
neve @ vocabulary as large as 3OO. Discrete voice systems 
senerailyu have 8 high accuracy rate (qreater than 75 
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Tere clthovgheaGhe aceuTrscu-is partially dependent on 


the size of the vocabulary, simularity of the sounds of The 
vuceouiAaTy words, and whether or not the systam is speaker 


ae The majority of fielded speech fTrecognition 
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Systems are discrete voice machines because the technology 
hos cimplier to produce and operate. 
ae Continuyuscyus Speech Recognition 

The most obvious characteristics of Senta mings 
Speech are the naturalness of the speech and the speaking 
Tete. Without artificial breaks in speech, speakers can talk 
eee oT aro, LOO to SOO words per minute. iLRer. 2: p. 
66] In continuous speech, beets Girticul > .tbomedetermine 


where one word ends and another tegins. Additionally, the 


SCHuvuttic patterns OF \WORdS” XO eg Tes Cer ee eee 
gepenaing on the context in which they are used. Vocabulary 


= bic impact on the complexity of these systems. 
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compact representation of acoustic patterns 
an Words, anc techniques for reducing searches by 


onstraining the “Set Gt possible @m@erdis "hat can e@eecur 2.54 


f*} 


Giver point assume added importance. 

Dunamic programming foe an cneT  amech od “Vor time 
elianment that is used in discrete speech recognition, but 
Bist greatiu enhances the capability of continuous speech 
Peooam i tl ome Dynamic programming allews for nonlinear 
tranztlation af speech, thus achieving better interior 


matching of the words. It determines how to time align the 
Segments of the Teference and spoken utterances. The process 
computes many mere combinations of time aiignments between 
‘eaference end spoken utterances than the static matrix 


ignifaicantly impreaves Tecounition resold. 
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TuT multisyllabic utterances. TRef. 4755p oOl 


Efficient search seem reducing algorithms can 
ie We de reduce the computational burden of matrix 
comparisons. These algorithms are made possible By 


representing words as strings of phonemes. Two advantages of 
TECOGNiZing phonemes in & voice Tecognition system are (i) 
selective recall of word ptTrototupes and (2) reduction of 
memotru requirements to store word oprototypes. Selective 


TecSsii reduces the number of pTrototypes that need To be 
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processed By keying on phonemes and using Piven —- 4 OTM 
words; algorithms can be developed to procets templates in 


am order more likely to result in an earliet match. Thus the 


USS of phonemes, or other sudword prototypes, allows 
comeETession cf ithe parametric representation oa word 
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SW we GUS “dimamic control to search eonly a 


THhOwtoTrd PTOGOtIupe dicgg@aenary. uolRefoees: p. 457 0.4 
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i “ White states CRet. ze D 491 “4G solution to the 
fumcgamentai probdiem of locai acoustic ambiguitu is to have 
Modis of Speech sounds at ail ieveis (phoneme. sullabie, 
Seeear eras), snd @ Sstraiteigu allowing lower models to cail 


om higher models to resolve ambiguities and 4 strategy 
ali:aguwing the higher models to cail on Slower models  “Cto 
reavest further analysis." These methods, which are not 


Svailable yet. would increase accuracy and consetTve 


G& meathod used in current COR Pinu S US speech 
fe eteeet ia eawinment is the use oF grammars to limit the 


* of wtutterences from bh ett the machine must choose 
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on previouslu recognized utterances. & grammar is a 
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representation of allowable word sequences in a state 
diagram composed of nodes representing a word or group of 
words and the sossible transitions between nodes. This 
metnhad is vertu efficient in a& system with a iimited 
vecetbulary, and in which strictly formatted sequences of 


Uerds ere required. CRef. 2: p. 32] 
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a “REVIUUS EAPER ies “GCenmowe reo 

There have been &@ number sf studies and erperiments 
cunducted at the Naval Postaraduate School concerned with 
the applications of vseing voice recognition technology iin 
command and control tasks. Some ot these experiments and 


Outcomes of telated studies will be reviewed briefly here. 


The objective of one experiment was to try to determine 
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re Bossibilitu af achieving speaker independence using 
apeaker dependent voice recognition equipment. The results 
snowed about 99 percent accuracy when the subject ’s voice 


stcerns were in memory along with those of fTOUTr Stier 
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Users, ans approximately a GSS percent accuracy when the 
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subyect’s voice patterns were not in memory, hut those of 


Four other users were. In this experiment, ail subjects were 
consideres exserienced in using the voice recognition 
aquipment Cet. 6] In a foelvou-on eo this, ancther 


experiment tested the recognition acetvracu with first. tame 
(maivels users of voice recognition equipment. Again the 


cterns of four independent speakers were in the memory of 
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sSpeaier dependent equipment. In this case 2 96.85 percent 
recognition accuracy was achieved, showing that training and 


practice may mot be required for voice applications. [kef. 
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Another study was made to determine the effects of 
feedback on the performance of voice recognition equipment. 


Feedback 15 used to indicate if a misTrecognition oT 
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emer cOecnition occurs. It was found that feedback has a 
amitea eftect on performance. Subyects not accustomed to 
secoack Teaduced errors by five percent when feedback was 
introduced. while subyects used to feedback had about five 
oetrcent more errors when feedback was reduced. (CRef. 8) 

une study attempted ta determine the effect of operator 
mental loading on voice fTrecognition performance. Besides 
being required to Trepeat words opresented audibly via a 
oy Sub yectTsS were subjected to varicus déegrees of 
mentvei loading by being required to make decisions of 


ing gecress of difficulty and make o response at the 


sam2 time A General Dynemics Response Analysis Tester 
CRATER; Model 3) was used to simulate the opetator mental 


Peo momeeesults ™ showes that vo1ce recognition performance 


Cearaded as mental leading wes imposed on the subyects. 


There were two experiments conducted to see if the use 
or Masks degraded voice recognition. Disctete speech 
t wes used in both cases. In the first. there wes an 
increase in recog hai on errors made when using a 
stencgraphers mask, but data showed this was primarily due 
to the inexperience of some subjects in speaking into a mask 
CRef. 10]. In the second experiment with Atmy protective 
(gas} masks, there was more serious degradation. The authers 
felt more research was needed in the placement of the 


microphone in the mask, and variation of word boundary 


HarTameters to nelp alleviate breathing sound problems which 


UsSuaily occut st the beginning and endina of Words. (Ref. 
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There nave been two experiments comparing the use of 
voice recognition equipment and manual typing to enter 
commands into a computer. In both cases discrete speech 
recognition equipment was used. In the first experiment, 


Subaiecrcs Ffolioswed a fixed scenario of instructions in which 
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LMey actessed the ARPANET and then performed a prescribed 
set of tas¥vs. The ARPANET is a2 large distributed network of 


emputers which are located around the United States and 


tl 


otner countries. PrioT to this@esper imei SUE jects Some 
am average of three hours of training on using voice 


ystems. Each subject performed the Seenariawrour Times aie 
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ch method of Input, With elf Veena vomeeowiInpus + ites ene 
the other naif using keyboard entry first. A secondary task 


igned in which subjects were to use any free time 
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curing the sessions to transcribe some data by hand onto 424 
e 
data sneet. The results of this experiment showed that: 
(13 Voice input was 17. SANfasterstian menval tising ina 
(2) Manual tupina input had 182.2% more entry errors. 
(3) Voice input allowed subyects to transcribe 25% more 


information than during manual input. 


The results were ali statistical tiie wae ae Pee OO ona 
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demonstrate that it is feasible to use current commercially 


available voice recognition equipment to run standard 
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: Discrete Speech Recognition Equipment 
The Threshold Technology, ae. Model™ %400 Voce 


imition system (hereafter referred to as the T6OCQ. or 
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Thresnoid} was used in this experiment to input discrete 
Yoicea commands. With several added memory devices, the 


~Sstem had the capacity to store up to 256 utterances 0.1 to 


= seeonds Jone. The system also contained 2 magnetic tape 
cartridge unit ouhich allowed the subjects tae record thers 


imdividvaei voice patterns and NWISS commanas after they 


initislly trained the machine. All subjects were assigned 


aft 


ividividuai tepe Ther, when the subjyect came back to use 


the T&é00 for NWISS sessions, the cartridae was read back 


imtoo memory and the subyect was ready to give voice input 


cammands. Gein could be manually set tao help overcome the 
interference af background noise. A Shure SMLO headset 


Microphone: svunpplied as standard equipment with the T6000, 
wes used 25 the input device. 

If; this experiment. we used the unbuffered mode 
which means that as the voice recognizer accepts a voice 
input. the word er phrase defined  +ftor Jthe uUtzerance Gar 
Tecagnizes 16 displayed on the screen, end an  ASCii 
character stream 1s immediately sent to the host computer 
without anu verification by the user that the voice input 


was correctly interpreted. This allows for the paessibility 


thet Che voice recognizer identifies something other than 


what you actually said, and therefore transmits the wrong 
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However, the voice input stream is Tetained in 
iffer of the host computer until a complete NWISS command 
25 Entered and 8&8 caTriage return character is issued to 


miso e ct the Compe ter to sct Upom the order. Until that tame 


the user has the option to delete portions of the input 
stream end repeat the desired input, or to cancel the entire 


Lream by issueing a "Control K" character and starting over 
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Centinucus Speech Recognition Equipment 
The Verbex SOOO voice terminal was the system used 
in this experiment to input continuous speech commands. The 


system 15 designed with a feature to automaticaliy set the 


im Gy sampling the speaker’s voice volume and then 
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Meximum vocebulary capacity of SSO words/utterances spread 
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twenty grammars. If the vocabulary is 


cantained in &@ single grammar, voice input can be in the 


fu 


[om oF maturally spoken stream of words, numbers, or 


phrases, with mo pauses. An example of this would be the use 
of continuscus voice Tecognition equipment to input zip 
codes. However, if more than one grammar is tTequited, a 


pause in speech input must be made when switching from one 


GgTammsr to another. There is a finita limit to grammar size, 


2? 


baeoead on the total number of words and the complexity of the 
made transiticn network. This is necessatyu to allow the 
system to fTremain “real time“ in terms of computation speed 


stored voice pattern requirements. Thus at any instant 
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the system is dealing with a subset of the entire grammer. & 
Shure SMiz&A headset microphone. supplied as standara 
Beauipment with the Verbex, was used as the input device. 


In addition to the Verbex 3000, the system includes 


fat; 


Soeech Aanplication Develscpment Systems (SPA4DS) which 
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Gus the user to program the voice terminal Go Pui 
Bet tise wd hor appl veatiomn, instead of purchasing customer 


engingering services from Verbex. It is this system that 
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iowed Lombardo to develop 4 program to use in applying 
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vo ice commends to NWISS. CRef. 14) 
One feature of the Verbsx 3000 is a smali unit that 


spomote the user for the proper command. Thus when the NWISS 


brought on line, it prompts aie “Geer to  entermen 
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Niviss command. When the first phrase is entered by voice, 


earches ror the proper grammar, and then prompts 
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the sustem 
the user again to enter any of an aliowable subset of 


ohrases to continue or complete the order. When the commana 
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is completed, it is automatically transmitted for execution, 
and the system again prompts for a new command. As with the 
FEQO, the Verbex user may cancel a partial stream by saying 


"Control KA and estore sever. 
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The tTemeinder of the equipment used in playing the 
wargame were keyboard terminals and the VAX 11/780 computer, 
om which the NWISS software resided. The Verbesx and 
mreshkoid speech input systems communicated to the VAX 


meee oui aiegler aDM-=-31l termineis. A TLigital VTiG2 terminal 


was used €5 the manual keyboard input device. Digital VT100 
terminals were used at all positions as status boards for 


the MWISS sessions. Ramtek high resolution color graphics 
Lerminails were used to display the game scenarios to the 
epee ceseeb) Sach position. The configuration of the lab is 


PesepTioea if Chepter Vi with a figure of the lad layout 


A cassette deck was used to play a tape ta introduce 


m7O18e aguTinag the NWISS sessions. Elgivy ineh, ,eignt onm 
sQeavers were nlaced at each position. Ranacom crowd naisa 
vee recatded on tape at a gathering of about thirty people 
SSaeeenoretenveTsationally in an euditorivum. 
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SUMMARY 
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Irn this chapter we have discussed some commoniy used 
terms and have described differences between discrete and 
continuous modes of voice input. We have reviewed some 
general theory used im design and development of automatic 
speech recognition systems. Previous experiments with voice 


technmologu coanacucted at the Naval Postgraduate School were 
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‘scribed inf order to give some beckground on what led up tao 


eriment, and possibie uses and limitations of voice 
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-cchioelogy applicati@ms in the military. Finelly, a 
eoctTiption was provided of the actual speech machines and 


Ooriated equipment used in this experiment. 
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id iHe NAVAL WARFARE INTERACTIVE SIMULATION SYSTEM 


This chapter contains a brief description of NWISS 9 and 


UIT it wat used in support of this experiment. eG alesis 
inciudes the scenarios developed for the experiment, Chet 
Seo al i Stion, and the measures of effectiveness selected. 
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used ta collect data and compare the three 


comneng input technologies are also described. 


MWwiSS is, as its name implies, an interactive simulation 


Pregeeat Wartare. Its mission was originally to train senior 


Nevel officers in force-level tactical decision making and 
manecgement af command and controi fCRef. 15: p.i-id. At NPS 
NwitS i6 resident on a VAX 11/7780 computer. The peripheral 


YTiGO/1OL terminals and RAMTEK graphics terminals provide 


iMipuertr/output modules when grouped together. Volce equipment 
wes used as "front-end" devices in two of the modules. Ali 


Imputs and outputs can be accomplished by one trained 
subyect from one module. NWISS was loaded, controlled and 
monitored from a fourth module during these experiments. 
The surpose of NWISS is to stimuiate a stressful environment 
ttat allows the user to intervene through use of NWISS 


commands. From the user’s standpoint, it has a large 
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tulary of commands to change tne waeu OF Ui eee 
vation in the battie area, cause changes in the course 
speed af vehicles in one force, and launch air assets 
user defined weapon loads and missions. NWISS alse 
plays the status of the user’s forces on @& variety of 
Selected dispiays. for this experiments each  stetron 
istedad of & command input terminal, a force status 
minai and &@ video monitor to graphically display force 
GSition anda tent mee 


MHiSS wes used to support this experiment because i¢ 


rs the ability te present identical taetical sitvatiug. 


fo three isolated subyects at the same time. This allows a 
Side by $iqcEe comparison of the command input methods chosen 
tu the isolated subjects when confronted by identicai 
tactical stimutlr ain @ controlled environment. It was also 
Zelected oecause 1t¢ i¢ avaliable at NPS and s&s pool of 
treined operators was at hand. 

The most important eTritverian, thadgin 2s NWiSS = laa 
Oe See tT ae If &@ very small vocabulery 1s chosen to run an 
Sxsperiment, the subyects may become bored through repetition 
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the same commands. Appraximately 100 words that are 
ssary to conduct 4 variety (oF miss loncead1 thea! on sence. 
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arrier battle group were selected for the voice input 


nologies. Twenty commands related to control of the 
ical ferce aisplay were also included. Tables of the 


lete vocabularies for the three input technoiogies are 
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tee sdead in Aopendix A. These vocabularies were determined 


Meeescao vente anc were limited by the size of the memory of the 


Veroaay sustem. Using the forces and commands in these 
Subsets of tne NWISS dictionary, four scenarios were 
fesioansd tao stimulate the subjects. 


im @li four scenarios, the Orange and Neutral force 


Sctirons were prepragrammed as were the Grange defensive 


Gia cire ane Trader emmisions control (EMCON?) ‘status. No 
Orange asircratt were used because introducing them is more 
comple, than required tor this experiment. The three views 
oF che worid fot the Blue forces were on approximately the 
same latitude and separated in longitude ty approximately 


SO mauticél miles. 


since NMISS is limited to a maximum of G1 X sets of 


fearces, the Orange and Neutral forces were structured as ane 
,aéu: Sach. There were three sepsreéete views for the Blue 


forces which allowed the subyects to control their own ships 
Aid ulaneés. The details of the process of assembling these 
Various views and overlapping them are not included in this 
fpader. They are explained in some detail in an NPS thesis 


Bueeuensmand Brown. CRef. 16: pp. 25-34] 
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& totel of four scenarios wete designed so that the 
Subjects would be placed in situations that required use of 


the available NWISS vocabulary and to stimulate different 
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rerponses This section describes the basic opening force 
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FC: hwof the scehearios and shown ia aa 
WUSELON briefina for that scenario. The situation 


UTLerings given to the subjects are in Appendiy B. 


Scenario A includes twelve PSC aircraft, an aircraft 


carrier, two frigates, and an attack submarine for the Elue 
TOTCES Grenge ferces consist of a catTrier and $1: sutface 
cumbetants. The Orange farces are escorting eight Neutral 


merchant ships in 8 convoy that is moving south as shown in 
Figure oe ie In this scenario. hostilities have been 
declared between Blue and Orange, and the objective is _ to 
rdentify and destroy the Orange escort vessels without 
rhe merchant ships. The merchant ships ate 


reportedly carrying nuclear ang chemicaei weapons to an 


=. SC enarices 
Blue forces for Scenario B are an aircraft carrier, 
two F2C’s to provide Cangeting information, ana tus 1 igh 
of five vattach aircraie: Twenty merchant vessels are at 
enchor outside a fetitieus Sherbanee focus ci vaitiapgeee. 
un Peed’ their we ead az Four Orange surface combatants are 


positioned in a line between the Blive forces and the 


merchant ships to defend them from attack. In Chis 
Si yuese 1 Ome hostilities have not been declared between Blue 
and Grange although tensions ate high. Blue’s mission 15 to 
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Glanmés tG conguct 4&4 strike on the merchant vessels. The 
Yui1es oF Engagement do not allow the Blue forces to FiTe on 


Orange until Orange has fited on them. From the opening 
Positions as shown in Figure 3.2, neither the merchant ships 


ior the UTranage combatants are visible to Blue, but the 
Mission is directed to fliy to the north. 
oe Acenariac € 

Scenario C presents a@ benign environment in which 
Blue tasking is to locate Gnd identity ae mane olenemes oe 
possible and designate them as Friendly, Neutrai or Enemy. 


Glue has one PSC and four Fi4A aipereft ta acecomeli sh cime 


Mission fm tetal of thirty ships and submarines are 
scattered over an area approximatelu 406 x 500 nautical 
Miles a5 snown im Figure see Combatants for both Blue and 


Uranue 85 weil as several merchant ships and fishing bosts 


are in this tTecteangle. The subjects were told that they did 
mot have control of the Blue surface forces in their area. 
This was an artificial lw amprodmceed samme trainitete 8c comp mee 
the tasks of Location and Sidemer weacvan. 

s. Sather Ol) 

The final sitwation pivtted tuo carrier battle groupe 
se approximately equal strength against one another. 
Slues’s task was to locate and identify the elements of the 
Orange battle group. Hositilities had not been declared but 
the subjects were given wice discretion to respond to any 
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the Urange carrier battie group to the mewen cine 
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with Orange headed north and Blue to the 
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she subjects were familiar with the play of NWISS and 
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Tiencea shortcuts in operating the game. Some 
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ds Such as changing the scale of the display are easy 


LG) ec come iich and Trapidly achieved. This 15 one of the 
Teasans we chase ta tell the subyects 4a false set oFf 
meacures of effectiveness. When the subyects received their 


mission briefing prior to each session, they were told some 
o> the measures of effectiveness by which their success o7 
FeiiuTte would be determined. Imciuded were such things a¢é 
the mumber oar merchant vessels sunk or damaged in Scenario 
SB, tne number of shigps correctly lotated aaa identified an 


amc the number of Biue forces lost to hostile 


i} 


scasnaria 
rite in Scenario A [See Appendix Bi. In e@ach case, the 
Subject was urged to accomplish the aoal before time 
expired. These MOE’s were not used far analysis, but were 
intended ta stimulate the subyects ta try to achieve a 
meaningful goal. 


The actual measures of effectiveness are Telated to the 


number of NWISS commands entered and their accuracy. The 
number of cammandads entered is the most positive indicator of 
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the input technique. Weer Cl Le titsceecine. + oOUT 


Uidg provide the opportunity to use many of the 


available in an environment complicated by the 


LGic@eeanmd noise levels. Times pees site was 


on the subyects by the schedule and by the actions 


LtateT Ss. Since all the trials lasted the same 
game minutes, the number of commands entered is 
aie 
1 


he number of commands entered 


een increased significantiy By the savvy subject 


the display radius numerous times. the brierina 
sed Fficvitious MOE’s effectively removed this 
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Faormance. Further, if ail that We Ss gesitred for 


ig was the rate of data entry, very Little 
Pemeis Needed fer the keyboard station. Piet 
e calculated from & Trepresentetive typing speed 


ve. We were interested in the environmental 
the input methoas in a command post setting, 
commands to forces are mot necessarily given in 
stream. 
can be measured to some extent by the number af 
omtrol K" cemmand is used to delete an attempted 


wevenr, there are some limitations to using 


as a measuTe of the number of errors made by 


ects. Since it aborts the current command and does not 


the “Control KA” could be viewed as a change oF 


subject’s part. From experience using the NWISS 
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dame. Enough: zt 1S jyust as easy to enter a completed 


mG issue & subsequent correction es it is to abort 
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here sre siso three equipment related factors thet 


coset Ce +TOM the use of “Control KA" as an error counter. 
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continuous speech fTrecognition system, after 
the  keyocard has been used to enter an NWISS command not 
imclused in the 111 word wocebulary, the “Control K" command 
oc necessary to return to the beginning of the command input 


njenae © Therefore, this use of met) tr One (ty does noe 
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imgicate an error, but the correct use of the system. 


f 


could be that the subject’s utterance was “oe 
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emized Gu the system and the subject elected to enter 


the cfommans manvally. The result is the same as in the 
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Case: the ebort command must be used to return to the 


staTrc of & mew commend input sequence. 
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both the Threshold discrete Tecognition system 


ea ie the keyboard have the capability to backspace and 
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errors in any command without deleting the whole 
COmMmManS. Se 1 16 possibile that alli the subyects in these 
Two sroups would not need the abort command except to 
indicate a change of mind. However, using the backspace 
function to make corrections involved using time that could 
otherwise be used to perform a "Control K" and reenter the 


ommand. With eli these factors eGonms toerec et 1eo) Conmcre. KS 
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command provides the best measure available of errors made 
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easuTs as the number of commands entered. 
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Meee COLL GCt:6n Was accomplished through several utility 
Tens of the VAX 11/780. Alli commands entered by the 


subjects’ command terminais were sent toa separate 
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in memory as was the record of the umpire’s game 
Ol terminel. THIS ,pmecess set up a record of all 
vwitu oy the subjects and kept a historical account of 
engagements, weapon firings and resuitant Gamage 
isyed on the umpite’s terminal for further study. 
The data files for each of the input methods were then 
ered bu another VAX utility program to output the 
tral K"“ abort commands. Tre printed outputs were then 
counted to determine the number of commands entered anda 
number of errors made. 
One additional comment is necessary fregarding how the 


oT: o¢ cemmands entered was determined. ite ae er ay 


rch sequence is widely used, amd is the most lengthy and 


icylt command. Since the aircraft launch sequence 


t least five lines to be complete and could be 


aborted anywhere within those lines, each line of the five 
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was correctly entered was counted as a command entered. 
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This chapter has discussed why NWISS was used for theirs 
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er iment ine lucene a sy (ppnler= te = description of NWISS 
analities. jhe four scenarios presented to the subjects 


gescribed, figures of the opening POSitions fopeat 
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given, and the situation briefings for the 
1eetTs are discussed. rine liu, the measures af 


ectviveness and deta collection techniques are outlined. 


Lye VOICE EXPERIMENT BACKGROUND 


fs PNTRQDUCTION 
Im preparing for the NWISS voice experiment, MeN’ 


reactors had tc be taken into account. The background of the 
Bubyeacts used in the experiment (their service, gaming 


experience, and experience with voice equipment and with 


eee > fae toa tbe considered. Other Geo16 ions made in th 
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xgerimenmtal design stage were concerned with time periods 


Gili 
mal 
a 


experiment, the number and ijength of runs, and 


rendomness of sequences, of environments, and of scenarios. 
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Heencr oon OF “SUBJECTS 
The bietcheround of the subjiects particapeating in the 


YULIG @xEberiment was extensive and varied. Ail subsyects were 


it 


aso ee Commend: ContTro!,. and Gommunications (C3) or 
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Pe oes Obetationg CUTTicuia. there was a mix of Army), 


eves ss emer FCorce’!7); Marineti?, and civilian(¢i) students. 


and five were female. 
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students had one year of graduate level course 
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nN 
inoerT vc Sion: Common courses included OS3404 (Man-Machine 
interaction), OS3603 (Simulation and Wargaming): and O584602 
(CB Sustems Evaluation). In O53404 students were introduced 
to comtinuous and discrete automatic speech recognition 


theory, and experimented with discrete voice inputs. They 
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6 introduced to the Digital VIiGO/71 Came evGeea 
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verminels ang Ramtek graphics screens in the War Lab which 
Would be used to play the war games. 

PS460sS iAnediwded instructions on gaming theory and 
experimentation. Students were rntroduced to NWIiSS, 
including its backgreund and symhologu. Further  instrwei we, 
Gfi discrete and continuous voice was given, and all students 
trained on either Verbex (2Onmcihnucus > or Threshoid 
giecrete} machines in the War Lab using a presctibed set of 
NMWiSS vocabulary. They had two practice sessions (three 


totel)s of entering common NWISS commands using the 
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Guguioment they had trained. [t should be noted that the 
enmcouire NWISS vocabulary could not be trained because of 
BGuUioment and design limitations. The course also included 
three SG hour sessions of groups piaying NWISS with a Sea of 
zr gcenaria, using keyboard to input commands. 

im OS46027 students had additional instruction on 
experimental design and analysis. They also had seven more 
three hour NWISS sessions inh conjunction with another 
eyoeriment. PT Non to the actual start af the voice 
experiment, subjects were assigned to one of the methods of 
imoput (Verbex, Threshold. or keyboard), with a rough balance 
of maval and non-naval students at the three positions. Each 
hed an additional one hour practice session using his or her 
assigned method of input using a practice scenario in order 


fo tregsin familiarity with the tTespective methods of input. 
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meee sl! Ceurses OF instruction, then, the total amount of 


Taeining related to this experiment was greater than sixty 


meours per student. 


ENVIRONMENTAL CONDITIONS 
ihe experiment was designed to be run in a variety of 
myirponments, based upon four different combinations of 


Md moise conditions. Additionally, four differant 


-o 
teal 
ay 
fli 


censrios were designed to be Tun so that subjects would 


SB different scenario during each of their four rTrun- 
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oceloru sessions. This design was chosen to complement 


he training and to further reduce the impact of learning on 


ne tTesuitant data. Otherwise, the aqata would be affected 
= supiects gained familiarity with a scenario, learning 
rom past mistakes, and thus increasing the numericai value 


- The measure Gor effectiveness. 
Teo er GOmMoinetions Of environment are as follows: 
Mae ouemagicoe,  mormal lighting: (benign condition) 


Soper ood 1.0mt 2 ho 
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fe -aren vigase;, NHhormal lighting 
Ste hemoise&, Low lighting: “most stressful condition) 


hese conditions were set to be representative of the 


myironment in a militety commend and control center, or in 
ther military settings where decisions have to be made and 
raers issued. The experiment was designed to compare 


ontinuous voice, discrete voice, and keyboard methods of 


-mput, ane to test if there Was an impect CM 2 yee 
for any oar the three methods as a Tesult of different 
environmental coanditions. 

Normal light conditions were defined as having overhead 
fiucrescent lights off, with track lights turned on and 


directed downward at a forty five degree angle toward the 


nearest well. Light measured near the game terminel screen 
wes apoTrorimately 1.0 foot-lambert. Low light  conditaane 
were set by turning aut all overhead lights: resulting. 
aporoximately OS foeoet-lamberts of light provided ~ 5s 7 


dame terminal CRT’s and the Ramtek qraphics screen. 


Moise was introduced by playing a tape of background 


cenvetrsetions or a large group of people. Speakers were 
placed beside the game terminals at all three gositions. Low 
noise was set at a 65 decibel level: and high noise was set 


ast =n #25 decibel level, as measured sone a Geseso.ec Suelo meee 
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subject ’s ear level. See Figure 9.1 in Chaoter 


Lab configuration. 
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i SCHEDULING OF SESSTONS 

Tne voice experiment was designed to be run during lab 
time reserved for the Q5446G62 ciass. Students in the class 
were assigned to ae of three sections based on their course 
schedule. This process was done by school academic 
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Sheet providing the capabilities of same of the forces, 


weasons, and detection systems used in NWISS. Because of its 


misc cer1catpon, this page 15 not included as an appendix. A 
veToal briefing on their mission was also given to subyects 
and amy questions were answered at this time. Subjects were 


Temianded to turn the handout in after finishing the session, 


G were instructed to a0 to theif positions to start the 


e. Completion of the Practice Sessian 
The WWISS game was GAilowed to ram ebe eon 
Peper imaceliys forty five minutes. At the completion, the 


Sub ,ects departec and, if another group was to come in, the 


EXpPerimenters prepared the leb for the next group. 
Sogditionaliy, copies of the photographed game scTreens were 


Ptarsceo Gut. in order to conserve paper: this was only done 


Soe <ne First few sessions sO that we could vetify that we 


Pe iGeeteNig sche Correct information; and to determine how 
iotrig the entire process would take. Printouts were not 


neeced of all the practice sessions because the data from 


these would not be used in the experiment. 
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The same procedure was followed in the test sessions as 
was used in the practice sessions. Prior to the arrival of 


the subjects, the proper NWISS scenario was initialized and 
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the woicée patterns for the subjects scheduled were loaded 
imte the machines (see Appendix C). The subjects were then 
brought ain and briefed and provided a three page handout, 
he Last two pages were the same as for the practice 


SCEmaT ic. Only the first page stating thre "situs ion. 
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ron, and MOE’s differed for each scenario (see Appendix 


kis 


D3}. The first page was the same for each position, except in 


one scenario when geographical coordinates were stated and 


li 


were different for each subject. The briefing included an 
Sexplsnation of peculiarities within a given scenario which 
she subjects would not normally have experienced previously. 


Following the briefing, the acttal MWISS session “seuan 


Gtiter the subjects went to their positions, the proper noise 
amg iighting itevels were set, and tne game was started. All 
She sessions were run for 3&2 minimum of forty minutes. 
Sergonnel who had repeated difrfrleumeri es were given 
4Htsistence af necessary. &t the conclusian of each run, 
cories of the game sessions were printed out and stamped for 
Tlassificeation The forty minute game time point was noted 
a7) the printouts, and the number of Commands and “Control 
ms" entered were computed. 
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planned, some rescheduling was needed due to 


eaonmnel being away on temporary duty stetus, or due to 
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ectea WNWISS geme probiems. Rescheduling of personnel 
ed no problems because each input method was used to 
am individual NWISS session. The proper scenaris and 


onmmental conditions could te 


set up for any individual 


boas Tequired. 
CUTTS Fey 
Peeceeeyes COR Mae described Lhe training of subjects in 
facet ont tals Gxperiment. All subjects trained with 
Pr tne continuous or discrete mode of voice input. This 
MEEEOTeCoT TG 8GCIVEN SeVvVeryEence en appreciation of the 
Pies Uaeyvoice inpltva ang to provide a sufficient 
from which to assign students to an apparatus they had 
rced om for the experiment. The chapter fTreviewed the 
ice ses¢ions that took place, and the procedures used 
gRWconcvct of thi experiment. 
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ee DATA ANALYSIS 


INTRODUCT LON 
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Byective of this experiment “was Gag eamuare woe 
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zay 
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erfectiveness of three interactive computer input methods 
under several simulated command post environments. The data 
coOliected was therefore analyzed with that in mind. The 
rmndgdevendent variables were two combinations of light and 
POLSE, ang tne dependent variables were the number af 
commands enteresae and the mumber of errors made. Each 


combination of environmental factors was analyzed from two 


Sspecrs: first, comparing the number of commands entered as 
S SoOsitive messure of effectiveness; and second, examining 


the rumber of “Control K’s" used gs Sanne tcaticn. 0 eae 


were committed by the subjects. 
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This chapter wili present summaries of the raw data, a 
ae a a expilsnation of the statistical methods used, and 
tTepert the results of the analusis in terms of the number of 
cCumnmands entered and errors mace. Through this analysis we 
haope to answer several questions. Pires t, 5, which 


environment does each input method perform best based on the 


two factors of light and noise? Second, within each 
envitanment, were there significant differences among the 
inpuy me tned. =| And finally, if the environments are 
rgnroTred, was there a "“best" input method for use in a2 
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ne additional question investigated that 
not related to the environments is whether or not the 
jects Exhibited 4 learning cuTve as a resuit of multiple 


¢ on the same input method. 


DeTA SUMMARY 

each command entered was a combination of NWISS words 
fed to accomplish a desired task. Again, since the 
raft launch sequence consisted of five separate lines, 
line carrectly entered was counted as a command. The 
tion of an error was taken to be the use of the 
wagieee ss oG abort a cammand for anu Teason. 

Tables 6. 1 and 6.2 summarize the mean number of commands 
Socio SEToTs for each imput method by combination of 


ronmental factors based on the data collected. Figures 


on 


and 6.2 Tepresent the same information in graphic form. 
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Figure 6.1 Mean Number of Commands Entered 
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MEAN NUMBER OF “CONTROL K‘S* 


Environment: Noise/Light 
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ahies 6 2 eng 64 present the average number of 


commends enterea and “Control K%s" used By triatenvumoe a 


Figures 63 and 6.4 summarize this data in graphic form. 


dats is attached in Appendix F. 


Tab ee. 6. 
MoAN NUMBER OF COMMANDS BY TRIAL NUMBER 
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of vaTiance techniques were used to compare 


rormance in the four environments for each input method 


d on the two factors of light and noise, and to rank the 
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im @ach environment. For most of the analyses, 


sample size 1¢ thirteen, Trepresentative of the thirteen 
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F threes participants. 
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e tuo wau analysis oat variance procedure on the 


Bal 
4 
el 


ite2p06Cr €adplicsetion program of the IBM SOSS computer was 


So determine what the environmental effects were on 
piece af equipment and to find out if they perform 
ET oT woTse in any particular environment of noise and 
c. The one way analusis of variance procedure (ADVO) 


h compares the means of several sets of data was used to 


compares the three input methods within each environment, 


Out Tecard to the environment, and to aetermine if 


Himaticant learning occurred through the trials. To use 


ts several assumptions must be made about the deta. 
Seecificaitu. it 15 assumed that each set OF Gata is 2 
Tandom sample, each sample is normally distributed end each 


roximately the same variance [Ref 20: 9) Pad. 
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In alii cases, the hypotheses were tested at a confidence 
level of TS percent. If significant results Werte Ghtaineducn 
the SG percent level, they sre slso reported. If the “fe 
ratio produced through the analusis of variance proved to be 
Significant at the 95 percent level, the Newman-Keuls range 
test wes apolied to determine where the differences were fear 


that Particular set of datawikef@sl: peers: 


ee Best Environment for Each Inpucetethiiag 

ceneTraliy, we expected the voice input methods to do 
retter in low light conditions and perhaps have less utility 
im &2 nigh noises environment. We also anticipated that the 
d would ¢e worse in the iow light environments. On 
the purpose crf analyzing the data, we hypothesized that the 
mean mumber of commands entered or the mean number of errars 
im €4ch environment would be the same. Based on the F 
ratios calculated after two way analysis of variance at the 
ercent confidence level, we were unable to reject these 
hupetheses for any af the three input methods. That is, 


mone of the three input methods tested was significantly 


batter or worse in all of the environments. 


= Bes tum Ut Pethoder ron Cach Environment 


This question i6 answered with the application of 


one usu analusis of variance testing with fout sets of two 
SimiLiar nNuil hypotheses. The two hypotheses were that 
Sitner the mean number cf commands entered or the mean 
mumcer Of arrers committed were equal for the three input 
methods, and they were compared in each of the four 
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Besed on the F tratic at a FS percent confidence 


Ly there is @ significant difference in the mean number 


STrvTGhe Mate in thee benian: mormal light/low noise 
ronment. In fact the F ratio was high enough to be 
Viuficant at the F9 percent level. The Newman-Keuls range 


confirms that the mean number of errors produced by the 
gata and Verbex groups were not sianificantiy different 


each other even though the raw data showed the keyboard 


hod with a slight advantage. Both were significantly 


from the Threshold score at a FF percent 
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im the low noise/iow Light environment there were neo 
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that Verbex and keyboard generally had higher numbers 


ommandge entered ene fewer errors than the Threshold 
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od. Similar resuits occurred in the high noise/normal 
t environment. In the worst case, the high noise/low 


eae tT Commenmt,; che mean nuUmser of Commands entered ror 
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ali three methods were much closer together which again 


Sak 


Ssuited in no significant cifferences. 


To SUMMSEPIZe tne TesvUlts fer che erect environments, 


ond tne benign environment produced statistically 
significant results for the meam number of errors produced 
by the three input methods. The Verbex and keyboard methods 


were essentialiy equal ana Threshold significantly worse. 
=p Best Overall Input Method 
For these comparisons the effects of the environment 


ignored, and the average number of commands entered and 


le 


o, 


"“foantrol K's" used from the four environments were tested 
against nuil hypotheses that the means were equal. Based on 


one way analysis of vésriance for the three methods, at a 95 


Sercent confidence level, these hypotheses can be rejected. 
4eTe;, too; the F rétios were significant at the GSO percent 


oa The aiternative hypotheses that there ‘Were 
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significant differences among both the mean number of 


commands entered and errors committed by the three groups 


The Wewmnan-Keuls test revealed two S1Gnit 1 came 
results of the 9S percent Cont idencese ve. FinTt. sts. fOr fen 
mean number of commands entered, Verbex and the keyboard 


were statistically equal, and both were significantly higher 
than the Threshoid method. Seconda, considering the mean 


Number Ofer COnLT Ole saweea, the manual entry method was 


Pee 1 CaNntiye owen than both of the voice entry methods, 


which were essentially equal. 
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2ne CUTVeS 
When considering the possiblity of a learning curve, 


whe hypotheses were that no significant differences among 


the mean mumber af commands entered or "Control K’s" used 
Sreevahaut the four trials. If there was a significant 
Givyrerence ang the number of commands increased ofr the 
Humber of errors decreased, then significant learning taok 
Pena. e Aithough there were no significant differences 
geprected based on the F ratios at & FS percent confidence 
level, same lesrning gid take place &s graphs in Figures 6.3 


= SUMMARY 

Eriefly restating the results of the analysis af thea 
Cavs gathered in this experiment, there were few 
Beer ee tice liv significant results. Nome Gr tne chreeginput 
meathods etesod out as being better or worse over all of the 
four combinations af light and noise. The keuboard and 


cantinucus epeech recognition system did have noticeably 
fewer errors in the least stressful Gt the -ouT 
environments. Without regard to the environment, the 
keyboard and Verbex groups were equal in their abilities to 
enter commands and able to enter more commands thatmene 


Vhpeshold qroup. The keyboard methodag also produced fewer 


rt 


ither of the voice entry techniques Finally, 
1G not exhibit s&s detectable learning pattern 


e of fouT trials. 
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Via CUNCLUSIONS AND RECOMMENDATIONS 


Pesed on the statistical analysis in Chapter VI, severel 


rr be oTawn. These conclusions do not 3 
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gGTee 
CUT Expectations in some cases. The purpose of this 
met 15 to explain some of the expected outcomes, their 
“Sl ouccome anc why they may have occurred. Also 
ucded ain this chapter are implications for data entry 


S665 in command ang contro! centers, recommendations for 


Te Experiments, and applications of voice technologies. 


St the cutset ., we expected the voice input methods to 
erforim the keuboard due to their more advanced 
Gu. iio Siso anticipated that the environments would 
some sianificant effects on all three input methods. 
Statietics in the last chapter do not beer out these 
Poa Presults. fn che woamoal{stoauwa@te the data did confirm 
Creme “Ueree ts Cid Mat leepm how to wse their input 
ees threavan the course of four trials. 
There were two reasons why the keyboard entry did as 
Cee oe 1G. First, most of the subyects wasted littie 
im moving their keyboards as close as possible to their 


input screens in the low light condition using the 


SS 


wideo display’s glow to Missist themeyn Seager ng the cadres. 
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Termine which subjects would use the keyboard, fate took a 


and. An informal survey of all subjects was conducted 
yuter the experiment to determine Telative skill levels on 
he keyboerd. The subjsects were asked to Tate themselves on 
he following scale: 


Hunt ana peck typist. 
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Ca: Able to use both hands, but must look at the keys. 


C3) SEX eT 2 eine suamim ieee the 2 sal, 


rhe pesults were that the Verbex group had a 1.469 average 
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level, Threshold a 2.00 and keyboard a 2.54 average 


evel, Keep in ming that this sutvey was puTely subjective, 
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does felb expiain why the keyboard group faiTtfead so 
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ime experiment does not provide concrets statistical 


idence Phat the keydoatda 1s supeT1SOT to voice 
epchnologies. The subyects in the keubserd group had 


@lativley superior typing skills, and yet the Verbex group 
Toved to be statistically equal in tGherevebiiity to enter 
ommands without regard to the ambient environment. None of 
he subjects in any of the groups could be considered as 


xperts in the art of voice command interaction with a 


omputer, while all had more than & rudimentary knowledge of 


the use of 3 keyboard. 
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The hackground noise level also did not heve a 


Significant effect on the number of commands entered nor the 
MUimust OF errors. We noted three factors that may have 
influenced this outcome. airs. the high noise level 
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merearwerU litrtie distraction te the keybGard operator in 
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tug vwoice inputs to a lesser degree. The keyboard 


acacers founda that they heard iliess of the Verbex and 
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hold subyects’ commands when the higher noise level was 


aoe 

[esas aises Fohe second point. the background noise 
apesented was essentially white noise, with veru few 
Gistinguishebila words or phrases durina the experiment. 
THLis may cr mey not be The case in a@ command center during 2a 


ina@ilu, the Verbex subyects may have experienced 4a 
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Tween environmental conditions during training 
and the actusl experimental environments, which could heave 


G tmeir performance: Oimimg training, access to the 
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SS Sls Wee not restrictied; so people could come and go at 
Wii Wchmouvgh thie nolsy, e@lir operated entry door. While the 
experiment was in progress, access to the lab area was more 
tightly controlled so that the noise of the door and light 
im the fouer erea would not affect the controlled interior 


environment. This should have resulted in fewer recognition 
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cerToobiems during testing, but there was no way of measuring a 
iy, ey Sie. 

here is one area where our expectations were borne out 
by the data collected: the subyects did not experience a 


amount ofr learning during the course of the four 
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trials. This Tresuit is desirable from the point of view 
that all subyects had reasonable familiarity with the task 
ae he gerrormed before the experimenta Situation is 
yTresented. We can conclude, then, that the combination of 
enrolimenr or the vocabularies, three practice sessians and 
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in another NWISS related experiment adequately 


orTepared the subjects for our experiment. 


PLICATIONS FOR DATA ENTRY METHODS 
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respite its age, the typewriter keyboard remains as a 


Pau age = metnod te enter vets or otherwise interact with a 
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modern computer. It 15 impervievs “Ge etn e else in eee 
environment and is veru nearly independent of the ambient 
lighting Conditions provided its operator 15 well Strained 
and has some means of feedback. The keyboard provides a 
high degres ofr accuracy though at times speed is sacrificed 
Pom aceuracu, Finally, the keyboard is user independent; 
thet is, anyone familiar with the  GWERTY Slaucut or Sean 
standard pattern can operate 1t with no additional training. 

Discrete volce recognition equipment, represented in 


this study ey the Threshotd Té00, 165 really only one step 


advanced from the keyboard. It accepts one utterance at a 
time versus one letter at a time. While this does represent 
@ large step, it does not approach the current state of the 
art in voice input devices. Discrete voice equipment is 
alsa independent of the light in the environment, but it can 
be sensitive tc the ambient noise. It does require some 
time on the part of the operator to train the device to 
Mmecoamize his or her voice. Without an adequate difference 
in the several samples given to the machine, a change in the 
stress level of the operator’s voice can cause incorrect 
moc oori £10n. The use of voice recognition equipment does 
have a significant advantage over the keyboard in that the 
Operator has the opportunity to perfotm some manual tasks in 
Priueon ir OmeanmGceracting with “the computer. This does 
present the possibility of overloading the operator, but the 
potential for performing manual tasks remains. 

Continucus voice recognition equipment such as the 
Verbex 3000 represents the state of the art in speech 
recognition technology. It offers greater potential for its 
users from the point of view that commands in the form of 
entire sentences can be employed. These can be recognized in 
considerably less time than is required to type them letter 
by letter on @ keyboard or to enter them one word at a2 time 
as in a discrete voice system. But it is also operator 
dependent and therefore not as universal as a keyboerd. 


Another shortcoming of the voice technology is the limited 
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memory space available to store a number of possible entry 
SEQUENCES. The continuous voice system, though, is less 
sensitive to the noise in the environment due to its noise 
cancellation feature. Like the Threshold system, Verbex 
offers the potential for limited manual tasks by the 


operator which could enhance his or her performance. 


Be RECOMMENDATIONS 
We offer these recommendations for further study of 
Yoilce LT BUG and keyboard comparisons. The Threshola 


discrete recognition system performed significantly worse 


than Verbex in almost all cases. For further tests, pit the 
keyboard against the continuous recognition device in 
similer environments and ignore the Threshold system 


altogether. Regarding the environments, it may be necessary 
to use background noise with recognizable streams of words 
Similiar to NWISS commands to properly test the concentration 
of the subjects. In addition, the training environment 
should more closely resemble the experimental conditions if 
at all possible. If time permits, we also suggest that all 
subyects perform trials on both the voice equipment and the 
keyboard to reduce the possible advantage of the keyboard. 
We feel that NWISS should be the medium used for. the 
/ 


experiment as it supports a large number of commands and is 


easily interfaced with the Verbex 3000 system. 
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SUMMARY 

[mes sdvent of Vergy Large Scale integration components in 
computer technology poses the possibility that memory 
devices can be made smaller than they are at present. This 
opens the door to more complex grammars in an automatic 
speech recognition sustem through increased memory capacity 
im less Space. From the results of our experiment, although 
the keyboard turned in the best results for the type of task 
tested, We believe the future for continuous speech 
Tecognition systems is bright. Investigation into their 
integration in intetactive computer programs for command and 
control center applications should be pursued. Thewreofrer 
the promise of tremendous benefits in applications where 
operators now using keyboard based devices are overwheimed 
bu the speea necessary to keep up with the flow of 


informaticn. 
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APPENDIX A. 


VOCABULARY LISTS FOR Ex*PER TRENT 


THRESHOLD NWISS VOCABULARY 


PROMPT SVT | MEANING 
O CANCEL (LINEREED) CONTROL K 
4 af aia FOR ADDRESS FORCES 
= CONSTELLATION CONNY SHIP NAME 
a PAUL PAUL SHIP NAME 
4 ADAMS ADAMS Sir Nan 
o CINCINATTI CINC? SUB NAME 
& BARBIE BARBI SHIP NAME 
4 ROANGKE ROANO SHIP NAME 
8 GOLDSBOROUGH GOLDS Sh L PS NatIE 
> MP ME 7 PSC ASW ACFT CALL SIGNS 
iO Sit SA7 SH2F ASW HELO CALL SIGNS 
ee VA? VA7 AS6E ATTACK ACFT CALL SIGNS 
te Ve VF 7 Fi4A CAP ACFT CALL SIGNS 
rs VK7 VKA7 KASD AIRTANKER CALL SIGNS 
i4 VW7 VW7 E2C EW ACFT CALL SIGNS 
iS 1 Pou = 1.2 PORCE COLLECTIVEM@ert Sich 
16 ZERO O NUMBER 
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SEVEN 


FIGHT 


Benes 


CENTER 


PLACE 


GRID 


Seah 


PUB 


ERASE 


Bsr 


DROP 


CANCEL 


GiRecLle 


XMARK 


RADIUS 


SeSsit FUN 
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BEARING 


eR CE 
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roan 
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ERASE 


ESM 


DROP 


CANCEL 


CBRE 


XMARK 


RADIUS 


POSITION 


NUMBER 

NUMBER 

NUMBER 

NUMBER 

NUMBER 

NUMBER 

NUMBER 

NUMBER 

NUMBER 

CHANGE CONTACT TO ORANGE 
ORANGE 

DEGREES TRUE 

OWN UNIT 

CENTER OF PLOT 

PLACE GRAPHICS ON PLOT 
PLACE GRID ON PLOT 
Gee SELECTED BEEMENTS 
ESM LINES OF BEARING 
ERASE ELEMENTS 
PLOT/CANCEL ESM LOBS 
ERASE TRACKS 

ERASE GRAPHIC ELEMENTS 
PLACE /CANCEL CIRCLE 
PLACE/CANCEL XMARK 
RADIUS OF PLOT 


LATITUDE/LONGITUDE 


se fay, 


NORTH 


SOUTH 


EAST 


Gru roe 
HARPOUN 
AT 
RANGE 
TORPEDO 
MARK 48 
ASROC 
ot Ee 
STAT ION 
COURSE 


PROCEED 


PERISCOPE 


DEP ata 


SURFACE 


APO 


ASO 


AUO 


ae a = 


CRUISE 


HRPON 


AT 


RANGE 


TORPEDO 


MKA48 


ASROC 


Sy a 2, 


STATION 


COURSE 


ed Cl lp B 


Gems SCOPE 


DER tr 


SURFACE 


LAT/LONG EG 43-30N 
DIRECTION 

DIRECTION 

DIRECTION 

DIRECTION 

ORANGE OR WHITE UNIT 
AIR TRACK 

ESM TRACK 

PASSIVE SONAR TRACK 
ACTIVE SURFACE TRACK 
ACTIVE SUBSURFACE TRACK 
mane ene Loe MIS TORE S 
CRerobeiiSsSlLe 

CRU@mae TlsolLeE 

CRUMSe MIS AT SHORE BASE 
DISTANCE IN NM 

ASW WEAPON 

SHERPSUE FIRED TORPERG 
Samer, ous FIRED GREE 
VELOCITY IN KNOTS 

POSIT RELATIVE TO GUIDE 
HEADING IN DEGREES TRUE 
TRAVEL CUS/DIST ORV reser 
SQewer teh PocePe Ver irn 
sUbpmeoe ih OIN FEET 


SUB TO SURFACE 


MISSION 


SEARCH 


CAP 


SiRIKE 


AIR TANKER 


SiR ihe -CAér 


BINGO 


ei neg ot ES 


EMCON 


STOP 


See 


RADIATE 


T&AE 


COVER 


ALTITUDE 


RBOC 


ON 


ibe 


WEAPONS 


TIGHT 


PRES 


ALL 


ATR 


DELSS Lan 


SURCAP 


SEARCH 


CAP 


> TR 


AIR TANKER 


STRCAP 
BINGO 
REFUEL 
EMCON 
STOP 
SILEN 
RADIA 
TAKE 
COVER 
ALTITUDE 
RBOC 

ON 

DECM 
OFF 
BLIP 
WEAPONS 
TIGHT 
FREE 
ALL 


AIR 


oS 


MISSION OF UNIT 


SURV7CAP MISSION 


Seance MISSION 


COMBAT AIR PATROL MISSION 


Sienne, M1 SS Tan 


AIRTANKER MISSION 


SUR LKECAP MISSION 


SGmine URN (0 BASE 


CAUSE AIR REFUELING 


Select EMCON PLAN 


COMPLETE LAUNCH COMMAND 


EMCON SILENT (CALL OFF) 


EMCON RADIATE (CALL ON) 


SHOOT AT TRACK 


ACFT TRAILS TRACK 


ScCeigeeSilGTUDE IN FEET 


USE SHIPBOARD CHAFF 


Var menbocy DeECM/ BLIP) 


DEF EEBe COUNTER MEAS 


Sor Wet (nBOC/DECM/BLIP) 


RADAR BLIP ENHANCER 


Sen ROE (FPREE/TIGHT? 


ROE NO USE OF WEAPONS 


ROE USE OF WEAPONS 
SNIERE S&T 


AIR SUBSET 


LAUNCH 


PHOENIX 


SHR IKE 


SPARROW 


SEDEWTINDER 


WALLEYE 


San 


BACK-UP 


ries 


LAUNCH 


ASE 


(CR) 
/ (CHAR } 


™, 
Cod 
. 


LAUNCH ACFT COMMAND 
ATTACK ACFT 

EARLY WARNING ACFT 
FIGHTER/CAP ACFT 

ATR TANKER ACFT 

LAND BASED ASW ACFT 

ASW HELO 

LOAD ACFT W/ EXPENDIBLES 
AIR DROPPED BOMB 

AIR TO AIR MISSILE 

AIR TO, SUREAGE »sMTLSSiise 
ATR Tamer 1) Somme 

ALR APRON ISStee 

AER” TOP SUREPACE MPSSTLe 
BLANK CHARACTER 

CARRIAGE RETURN 

FRASE CHARACTER (CTRL A) 


ASK FOR HELP 


VERBEX NWISS VOCABULARY 


PROMPT SAY MEANING 

G CGNTRQGL K CONTROL K 

i FOR ADDRESS FORCES 

z KITTY KITTYHAWK _ SHIP NAME 

S KNOX KNOX SHIP NAME 

4 MCCOR MCCORMICK __ SHIP NAME 

5 OMAHA OMAHA SUB NAME 

é RATHE RATHBURN__ SHIP NAME 

3 WICHI WICHITA___ SHIP NAME 

fs WILSG WILSON __ SHIP NAME 

9 MPS P3C ASW ACFT CALL SIGNS 
iG = SH SH2F ASW HELO CALL SIGNS 
| UV? -ve ASE ATTACK ACFT CALL SIGNS 
i2 VFO Fi4dA CAP ACFT CALL SIGNS 
ia ©=6VKO KASD AIRTANKER CALL SIGNS 
14  VWO E2C EW ACFT CALL SIGNS 

15 1 POINT 1 FORCE COLLECTIVE CALL SIGNS 
ig oO NUMBER 

7d NUMBER 

ig 2 NUMBER 

ig 3 NUMBER 

a NUMBER 

2S NUMBER 


7 


& NUMBER 


v NUMBER 

g NUMBER 

g NUMBER 

DESIGNATE CHANGE CONTACT TO ORANGE 
ENEMY ORANGE 

BEARING DEGREES TRUE 

FORCE OWN UNIT 

CENTER CENTER OF PLOT 

PLACE PLACE GRAPHICS ON PLOT 
GRID PLACE GRID ON PLOT 
PLOT PLOT SELECTED ELEMENTS 
LOB ESM LINES OF BEARING 
ERASE ERASE ELEMENTS 

cSM PLOT/CANCEL ESM LOB 
DROP ERASE TRACKS 

CANCEL ERASE GRAPHIC ELEMENTS 
CIRCLE PLACE/CANCEL CIRCLE 
XMARK PLACE/CANCEL XMARK 
RADIUS RADIUS OF PLOT 
POSITION LATITUDE/LONGITUDE 
Se LAT/LONG EG 43-30N 

N NORTH DIRECTION 

S SOUTH DIRECTION 

E EAST DIRECTION 

W WEST DIRECTION 


OS 


BUO 


Pla 


ERNE NES) 


HRPON 


AT 


RANGE 


TORPEDGS 


MK48 


ASROC 


Sra 


STATION 


COURSE 


rie OLS See. 


PERS CERE 


DEST 


SURFACE 


MiSSsian 


SURCAP 


SEARCH 


CAP 


STRIKE 


HARP OON 
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TRACK (ORANGE OR WHITE UNIT) 
AIR TRACK 

ESM TRACK 

PASSIVE SONAR TRACK 
ACTIVE SURFACE TRACK 
ACTIVE SUBSURFACE TRACK 
FIRE CRUISE MIS/TORPS 
CRUISE MISSILE 

CRUISE MISSILE 

CRUISE MISSILE AT SHORE BASE 
DISTANCE IN NM 

ASW WEAPON 

SHIP/SUB FIRED TORPEDO 
SHIP/SUB FIRED TORPEDO 
VELOCITY IN KNOTS 

POSIT RELATIVE TO GUIDE 
HEADING IN DEGREES TRUE 
TRAVEL COURSE OR POSIT 
SUB TQ PERISCOPE DEPTH 
SUB DEPTH IN FEET 

SUB TO SURFACE 

MISSION OF UNIT 

SURV/CAP MISSION 

SEARCH MISSION 

COMBAT AIR PATROL MISSION 


=TRIKE MISSION 


SAITRTANKER 


EMCON 


STOP 


STEER 


COVER 


ALTITUDE 


RBOC 


ON 


Dec 


GFF 


Bete 


WEAPONS 


TIGHT 


PRee 


ALL 


AIR 


LAUNCH 


Og a 


RADIATE 
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AIRTANKER MISSION 
STRIKECAP MISSION 

ACFT RETURN TO BASE 
CAUSE AIR REFUELING 
SELECT Enea PLAN 
COMPLETE LAUNCH COMMAND 
EMCON SILENT (CALL OFF) 
EMCON RADIATE (CALL ON) 
SHOOT AT TRACK 

ACFT TRAILS TRACK 
ACR ALP erebe IN reer 
USE SHIPBOARD CHAFF 
USE CRESE7 DECM/ SET?) 


DEF ELEC COUNTER MEASURE 


STOP? OSE” Creee/7 DECH/ELIPF? 


RADAR BLIP ENHANCER 


SET ROE CrREE/T Teri) 


7 

O 

ir) 
I 


NO USE OF WEAPONS 
ROE = USE OF WEAPONS 

Sn ine om 

AIR SUBSET 

LAUNCH ACFT COMMAND 
ATTACK ACFT 

EARLY WARNING ACFT 
FIGHTER/CAP ACET 


AIRTANAER ACFT 


F3C 


Saher 


LOAD 


MK83 


PHENX 


SHRIK 


SPAR 


SWDR 


PESPLAY 


VA00 


PHOENIX __ 
SHRIK | 
SPARROW __ 
SIDEWINDER 


WALLEYE 
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LAND BASED ASW ACFT 


ASW HELO 


LOAD ACFT W/ EXPENDIBLES 


AIR 


AIR 


AIR 


AIR 


AIR 


AIR 


DROPPED BOMB 


mo AIR MISSILE 


fee SURkence Mi SSiie 


moa ReMi SSTiEe 


me ATR MISSILE 


To SURFACE MISSmeeE 


ACCESS GRAPHICS 


CALL FOR REFUELING 


KEYBOARD NWISS VOCABULARY 


COMMAND MEANING 
G CORMR 2 la CONTROL K 
1 FOR ADDRESS FORGES 
a 4AMERT SHIP NAME 
e BAGLE SHIP NAME 
= TOWER SHIP NAME 
2 MISAW SHIP NAME 
‘= NYC SUB NAME 
v4 FANNI SHIP NAME 
G WABAS SHIP NAME 
vi STGDD SHIP NAME 
16 Mes P3C ASW ACFT CALL SIGNS 
oh SH& ' GSHEF AGW HELO CALL SIGNS 
is VAG ASE ATTACK ACFT CALL SIGNS 
UE. (FE FiSA CAP ACFT CALL SIGNS 
i4 VK8E KASD AIRTANKER CALL SIGNS 
1, VG E2C EW ACFT CALL SIGNS 
as ie OLN 2S FORCE COLECEC? IVesGeacEe Sicn 
17 O NUMBER 
is i NUMBER 
i? 2 NUMBER 
20 e NUMBER 
24 4 . NUMBER 


chi 


& 


Zi 


S 


¢ 


DESIGNATE 


NEUTRAL 
FRIENDLY 
ENEMY 
BEARING 
FORCE 
CENTER 
PLACE 
GRID 


BEOT 


DROP 
GaNGEL 
Cin eee 
XMARK 
RADIUS 


POSITION 


NUMBER 


NUMBER 


NUMBER 


NUMBER 


NUMBER 


CHANGE CONTACT DESIGNATION 


WHITE 


BLEUE 


ORANGE 


DEGREES MERUE 


OWN UNIT 


CENTER OF FLO 


PLACE GRAPHICS ON PLOT 


PLACE GRID ON PLOT 


PEG eSEERGHED SEEMENTS 


ESM LINES OF BEARING 


PLOT/ERASE SONAR LINES 


ERASE  SESrENTS 


PEBTACANCEEeS LOL 


ERASE TRACKS 


ERASE GRAPHIC ELEMENTS 


PLACE/CANCEL CIRCLE 


PLACE/CANCEL XMARK 


RADIUS OF PLOT 


LATITUDE/LONGITUDE 


LAT/LONG EG 43-30N 


Ore 


72 


rae 


CRUISE 


HRPON 


AT 


RANGE 


TORPEDO 


MK43 


ASROC 


PRECEED 


PER2SCseE 


DEPTH 


SURFACE 


MISSION 


DIRECTION 


DIRECTION 


DIRECTION 


DIRECTION 


TRACK CORANGE OR WHITE UNIT) 


AIR TRACK 

ESM TRACK 

PASSIVE SONAR TRACK 
ACTIVE SURFACE TRACK 
ACTIVE SUBSURFACE TRACK 
PLREwGCRVISE shes / TORS 
CRUTSE FilSstere 

Ce URIS S TEE 
SRUTSE@MES "Av onGRE BASE 
DISTANCE IN NM 

ASW WEAPON 

SHIP/SUB FIRED TORPEDO 
SHER SUE SE TREDSIG@RPEDS 
VELOCITY IN KNOTS 
POSITORELATEVE 10 CUIDE 
HEADING IN DEGREES TRUE 
TipeyveekCUS/DiSh OR POSTE 
SUB VO" PERTS@Sre DEFT 
SUB DEPTH INSEEET 

SUB TO SURFACE 


MiSsStOn GF CNet 
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98 


99 


SURCAP 


SEARCH 


CAP 


SiR. AE 


AIRTANKER 


SURCAP 


BINGO 


oS ae) 


EMCON 


STEEN 


RADIA 


TAKE 


COVER 


ALTITUDE 


Bia 


WEAPONS 


TIGHT 


fees 


ALL 


AIR 


LAUNCH 


AGE 


SURV/CAP MISSION 


SEARCH MISSION 


COMBAT AIR PATROL MISSION 


STRIKE MISSION 


AIRTANKER MISSION 


STRIKECAP MISSION 


ACFT RETURN TO BASE 


CAUSE ATR REFUELING 


SELECT EMCON PLAN 


STO Neo eeeNT VCaAkL OFF > 


EMCON RADIATE (ALL GON) 


SHOOT AT TRACK 


ACFT TRAILS TRACK 


ACE) ASTI TURE WIN FEET 


USE SHIPBOARD CHAFF 


Joo UR woGy Voc EleiP) 


Peme lee Guu PenamMaaAs 


See USeEetn DOC 7 RPECM/ELIP) 


RADAR ELIP ENHANCER 


SET) ROG MPR EE, TrerH1) 


ROE NO USE OF WEAPONS 


Roe USE OF WEAPONS 
ENTIRE sai 

AtR SUCSEw 

LAUNCH ACFT COPMAND 


ATTACK ACFT 


LOS 


100 


id 


LOAD 


MK83 


PHENX 


SHRIK 


SPAR 


SWDR 


WALL I 


Suen 


ATTACK ACFT 


EARLY WARNING ACFT 


FIGHTER/CAP ACFT 


AIRTANKER ACFT 


LAND BASED ASW ACFT 


ASW HELO 


LOAD ACFT W/ EXPENDIBLES 


AIR 


AIR 


AIR 


AIR 


AIR 


AIR 


DROPPED DCMmE 


TO_ ALR MESS ERE 


TO.SURP ACES SSs Tre 


TOW ALR GSS bie 


10 AIR MESS Ttee 


TO SURBGECE Missle 


COMPLETE LAUNCH COMMAND 
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APPENDIX B. 


SCENARIO BRIEFINGS 
PReei. Ce. ScCeNar 1 
SITGSTIGN: Relations with Orange have been deteriorating, 


md hostilities are expected to break out soon. An Orange 


th 


ifs 


AG has been reported to the north. 

een yY FORCES: NOU have sccucarriermis battle. gio picu th 
associated aircraft. 

Pmeriy PURGES: Strength of Orange forces is unknown at this 
Time. 

MISSION: You are to tJlaunch aircraft and search for and 
identify anemy forces. Weapons will remain tight unless 
fired upon; then you may respond in kind. 

ig |G fee Pes ye 

-# of aircraft launched and loaded (HRPON’s should be loadega 
on FS3C’s and 4456's} 

-# of enemy designated 


—-#% of enemy forces acquired, hit. and damaged 


NEGATIVE 
-# of enemy forces not designated 


—-# of friendly forces damaged/iost 


OZ 


SCENARIO A 
SITUATION: Orange is escorting a large convoy of cargo ships 
conteining nuclear and chemical weapons to one of its 
@llies. Hostilities have been declared. 
FRIENDLY FORCES: You have 2 Knox class ships, a carrier with 
Sit assets, and e& flights of 4 P3C’s available. The F3’s are 
loaced with 4 Harpoon missiles and 4& MK46A~ torpedos. 
Additionaiiy, there 1s &@ Los Angeles class submarine whose 
HoSsSition is north of tiie convo. 
ENEMY FORCES: Orange has approximately & ships escorting the 
je ome. There are approximately 10 merchant ships in the 
convoy. Tine are no enemy aircraft in the air, nor are they 
expected to launch any. 
Miss BN; Destroy the Orange combatants using available 
assets without damaging any cargo vehicles (the type of 
cargo of the merchant ships makes any damage to them 
oT Ohibieive). 
eel ee PUStaTlvs 
—-# of weapons fired during play 


-# of combatants acquired, hit. and damaged/sunk 


NEGATIVE 
—-# of merchants hit/sunk 
—-# of Blue aircraft lost 


—~# of Blue vessals hit/sunk 
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SCENARIO 8B 


SITUATION: There sre approximately 20 merchant ships at 


anchor waiting to unicad at a pert in one of Orange’s newest 


Seewenies. heir cargo inciudes POL, Spare aircraft parts, 
and ammunition to supply the militia of the colony. 
Diplomatic relations with Orange heve been  Oobroken:, and 


Nhastilities are imminent. 

Meese y FORCES: Your forces imelude 2 P3C’%s loaded with 
Harocons and MK46A torpedos, and two groups of 5 ASE’sS which 
are also loaded with Harpoons and torpedos. You also have ¢ 
ceTrrier availabie to launch additional forces. 

ENEMY FORCES: Orange is prepared to defend the merchant 
ships, but bad weather has not allowed us to verify its 
location or makeup. 

MISSIGN: You are to destroy 4s many merchant ships as 
possible, minimizing your own losses. If there are Orange 
forces in the area, you may attack if they fire on you. Your 
srimsary mission is the merchant ships. 

Rae : PUSiITIVE 

-# ofr weanons fired 

-# of merchant ships acquired, hit, and damaged 


-# of Grange combatants acquired, hit, and damaged 


NEGATIVE 
-# of Blue forces damaged/iost 


—-# of merchant ships remaining 


SCENARIO C 


% 
bend 
4 


VATION: Numerous Orange combatant elements have left 
their potts and are patrolling merchant ship lanes. They 
have been running EMCON silent and our national level 
sensors have been unable to locate them. There are friendly 
and neutral forces in the area also. Tensions are frising, 
but no hostilities have been declared. 

FRIENDLY FORCES: Blue has 1 FP3 and 4 Fi4A’s launched with 
standard loads. There are other Glue forces in the area, not 
under your control. Merchant ships are not hostile. 

ENEMY FORCES: There are enemy forces throughout the area. 
Enemy aircraft are mot expected to be encountered. 

MISSION: Search for contacts in the area from 2JSN to 42N and 
from it74E to I1S06€. Locate and identify as many vessals as 
scassible, and designate them as enemy, friendly, or neutral. 
Weapons will remain tight unless fired upon. Center your 
display upon your P3. Use only the flight/aircraft and any 
active track status boards. 

rio ee POSiT ive 

-# of enemy identified properly 

—-# of friendly forces identified properly 


-# of neutral ships identified properly 
NEGA&TIVE 
-# of forces improperly designated 


-# of forces not found/designated 


SCENARIO D 


SITUATION: Last night we received word that an Orange SAG 
was 200 NM north of our position, and we have been steaming 
TG intercept and visually identify their forces. At this 
time we shoula be approximately 100 NM from them. 
Hostiiities have not been declared, but fighting 15 expected 
to break out soon. 

er teNbLY PORCES: Blue has a4 carrier with associated 
aircraft, 3 other ships, and a submarine available. 

ENEMY FORCES: Strength of Orange forces if unknown. They are 
not expected to have aircraft launched at this time. 

MISSION: Launch carrier based assets to search for and 
identify possible Orange SAG. Weapons will remain tight 
unless fired upon. Then action is at your discretion. 

ae pe S ieee 

-# of aircraft launches 

-# af enemy assets identified 


-# of enemy assets damaged (if fired upon) 


} 


NEGATIVE 
-#% of Orange forces not designated as enemy 


—-# of Blue aircraft lost 


—-# of Blue ships hit/lost 


(J OJ 
Lao NO Mee oO 


el coll ooo od 
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PJ 


LS 
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30 


SPEND TX ac. 


VOICE EXE BRIMENT TEAM eMEMBeR otk 


TEAM THRESHOLD KEYBOARD VERBEX 
i HENRY LARSON HUNZEKER 
2 BAKER VONDERSCHEER VANE 
= TURNER DIASE HARDEE 
4 BRENNAN SHELL WOLFRUM 
5 CHAMBERLAIN SULLIVAN ANDERSON 
& SMART BASKEYFIELD LANDAY 
i. AWIATKOWSKI YOUNG BOYD 
= BOURN JASKOT HAWRYLAK 
9 COX ASHBY MILLER 
id CARLSON PETERSON GREENSPAN 
ii CORBELL BERGER YEAKEL 
i2 AMBROSE JOHNSON BREIDERT 
Mee HUMPHRIES GATES TROY 
TEAM/SCENARIO SCHEDULE 
DAY TIME TIME TINE TIME TIME TIME TIME 721MeE Tine 
i2-i3 i2-i4 14-15 08-09 09-10 1i0-ii 14-15 i5-i6 i6-i7 
OC ery 4 2Pi 
GGT SP i $3 4Pi SPi SP i 7Pi 
NOV BP 1 GP 1 HH 
NOV 1OP 1. “area +H 12Pi 13Pi +t 
NOV ial $$ 2B2 
NOV 3C3 HH 4Bi SD3 6A4 fie 
NOV BD2 ?B4 SBCi 
NOV 10Di 1142 1004 i12B3 13¢C4 12Ae2 
NOV ibd4 $3 2AI 
NOV SB2 HH 444 9C2 6D3 7B4 
NOV BAS PAB 9Cci 
NOV {0A2 11Di 11B3 i204 13B3 i3Di 
NOV iBe2 34 2C3 
NOV 3D4 $3 4C2 9A4 6Bl 7D2 
NOV TEAM RECAP DAY 
NGYV ic3 4 2D4 
NOV SAI 3H 4D3 9Bi 6C2 TAS 
NOV $ BB4 7D2. tt 
NOV 10B3 t3Xv4 st 12Di i13A2 st 


APPENDIX OD. 


ADMINISTRATIVE BRIEFING 
- Subjects were to wait outside the lab until told to come 
im. This would prevent disturbing the lab environment if a 
gToup wes scheduled before them. 
- Sessions would last approximately fifty minutes. 
-— Voice patterns were already loaded for the voice users. 
Yeroex users simply had to set their gain to be ready to 
Suet vt. 
—- The neise and light conditions for the exercise were 
explained. a 
- The set-ups were explained. 
- Subjects were advised to ask for help during the practice 
session if they Tan into problems. 
Ser fevowece SUbJectsS still had trouble with inputs after 
three attempts, they were advised to revert to keuboard 
entry ta finish that command. They were reminded that they 
could retrain utterances any time there was not an NWISS 


=o 


iN 


sion in progress 

-~ Subjects wete asked not to discuss any scenarios following 
their sessions until the experiment was completed. 

- The group was reminded of their mext scheduted lab period 


for the experiment. 


APPENDIX E. 


SAMPLE TASKS TO" SE > PERFURMECD 


~fire hanpoon cmumise missiles from ad hegeee oe sie 

—-fire torpedos at tracks 

-change depth of submarine 

~chenge speed, course, or position of blue forces 

-~determine bearing end range from friendiy force to desired track 
~ga weapons free on enemy or all 

—~change emcon status 

-launch ana load aircraft 

-place circles of desited radius around a force or position 


~desiqnate tracks as enemy, friendly. or neutral 


APPENDIX F. 


INDIVIDUAL TEST SESSION DATA 


TOTAL COMMANDS Teta CONTREEFK “s 
Sree oelw ENVY ORD VER THR KEY oes) TR oA Y 
1 4 sf 1 26 oe, 20 tae 3 = 
Ep = = 69 a7 ey <0 = 3 
c c 4 40 =) SS, =) & 1 
CD 4 = a 66 62 20 3 Ss 
= A 1 Z e7 45 64 9 ae OS: 
5 = if Shs 46 4i 10 sla = 
c 3 = 4G Sy, es 4 i sl 
D “ 4 37 6i 69 7 2 = 
S & 1 4 SO 28 SS ee <4 a 
5 = = ee 38 29 12 i7 oo 
C S 1 47/ =O a6 8 25 10 
D “ 3 99 6i 62 & 4 16 
4 A + & Si, Fo. iis i4 8 i2 
ie 1 i 70 on 98 Le, aO 14 
S a 3 Bi 46 BF 7. 6 2) 
B 3 4 Si 7S emlie © ie 10 10 
5, a 4 3 Heep 43 ore ie, =) 2 
3 it 4 FOS ve Gi i0 42 O 
€ = = oid 24 34 1 i 1 
D 2 1 58 44 36 iS S| = 
é a 4 1 i100 76 8 Le 46 Ege 
EG 1 3 ioe Tae, 88 22 34 = 
© = 4 G7 68 PTs i7 a) 10 
D 3 2 94 78 rile: 7 34 , 
7. = E 4 Teds 4/7 119 10 32 2 
B Me 2 20 44 86 8 a 4 
C i id e\3, 44 92 6 19 7) 
D = = 60 30 i006 = 8 4 
& ~ 3 Ss 81 40 Sc 28 ile ie 
B 4 4 76 45 B2 Ls, 22 7 
e 1 2 EIY/ Ss 70 =| 20 A 
D = 1 &7 So 62 8 i8 UE 


2 A 5 2 67 
E 4 1 75 
© i 3 60 
D = 4 2), 
10 = & 2 a 141 
G S 4 76 
C 4 = 74 
D 1 i 2 
ii 6A = ae 60 
E S 3 oF 
C ~ 4 27 
D 1 2 G2 
if A 2 o ve 
BE SB: 1 26 
c 4 ic 45 
D 1 43 
i3 A 2 4 28 
B e = 04 
& 4 1 30 
D i 3 To 
GRP=GROUP 


SCN=SCENARIO 

ORD=ORDER OF OCCURANCE 
VER=VERBEX 
THR=THRESHOLD 

KE Y=KEYBOARD 
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